feat(web): audio playback (RDPSND) through an audioPlayback callback - #2020
meanaverage (meanaverage) wants to merge 1 commit into
Conversation
The web client had no audio output. An RDPSND client backend (16-bit PCM
at 48, 44.1 and 22.05 kHz) now forwards what the server sends to the
session loop, which hands it to the JavaScript callback given with the new
`audioPlayback(callback)` extension: `{ type: 'wave', sampleRate,
channels, data }`, `{ type: 'volume', left, right }` and
`{ type: 'close' }`. With a callback, audio playback is enabled in the
client info; without one, nothing changes.
Windows servers only start playback once RDPDR is up as well (FreeRDP
enables device redirection along with audio for the same reason), so a
device-less RDPDR is attached when no printer is.
GitHub Actions
Active
… #1977 - Sound in the web client is now Devolutions/IronRDP#2020, rebased onto master on its own, with unit tests. - Devolutions/IronRDP#1977 (another contributor's) turns the graphics pipeline on for every web connection and follows ResetGraphics: with it and our other pull requests, all GNOME and Windows suites pass without our patch 5, which goes when it lands. - The plugin sets the graphicsPipeline extension only when the IronRDP build has it, so it keeps working once that patch is gone.
github-actions
Bot
added
kind/protocol
|
This pull request may overlap with #1764. Both expose new RDP capabilities to the web client as WASM session-builder options: this PR adds an audio_playback extension plus RDPSND wiring, and This notice is advisory only. Automated review continues as usual, and how these pull requests relate is for maintainers and authors to decide. Note LLM-assisted content (no human feedback). |
github-actions
Bot
added
the
needs-review
|
Automated review will not run because this contributor is not yet eligible under the automation policy. Contributors become eligible after one qualifying IronRDP pull request is merged into |
|
On the overlap note: I compared the diffs. #1764 adds pointer rendering, desktop scale, performance flags and time zone options, and no audio. The only contact point is its comment above the |
This branch was successfully deployed
1 participant
Problem
The web client has no audio output. It only attaches a no-op RDPSND processor (to satisfy RDPDR's channel dependency when a printer is redirected), so the remote desktop's sound is discarded.
Change
iron-remote-desktop-rdp: a newaudioPlayback(callback)extension. The callback receives{ type: 'wave', sampleRate, channels, data }(interleaved 16-bit little-endian PCM in aUint8Array),{ type: 'volume', left, right }(0 to 1) and{ type: 'close' }. Playing it (for example, scheduling Web Audio buffers) is left to the embedding application, as with the clipboard.ironrdp-web: aWasmAudioBackend(RdpsndClientHandler) advertising uncompressed 16-bit PCM (stereo, 48, 44.1 and 22.05 kHz). It forwards what the server sends to the session loop, which calls the JavaScript callback. With a callback,enable_audio_playbackis set in the client info.NoopRdpdrBackend) is attached; with one, the printer's RDPDR serves, and the real RDPSND replaces the no-op one.Testing
crates/ironrdp-web/src/audio.rs: only 16-bit PCM is advertised, PCM blocks are forwarded with their rate and channel count, other formats are dropped, and volume and close are forwarded.cargo test -p ironrdp-web --lib: 33 passed. The 19clipboard::testsfailures are the same onmaster(they need a browser forJsValue).cargo xtask check fmt,cargo clippy -p ironrdp-web --all-targets -- -D warnings(host andwasm32-unknown-unknown),cargo xtask check typos, andnpm run checkiniron-remote-desktop-rdp.masterwith this change, the web client plays audio from Windows 11. Our end-to-end test plays a system sound inside the session and checks that PCM with a non-zero peak arrives. GNOME Remote Desktop 46 also works, together with fix(rdpsnd): echo the Training PDU's wPackSize in the Training Confirm #2019 (its Training Confirm check) and the graphics pipeline, which GNOME requires (fix(egfx): stop the desktop from tearing when the server resizes graphics output #1977).Prepared with AI assistance; I reviewed the change and ran the tests above.